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QFP  Architecture 
•  Ada/C++/C 


OFP  Architecture 


Boeing’s  Previous  System  Arch  Lesson  Learned 

Case  Studies 


•  Software  Modification/Maintenance  Costs  Are  a 
Significant  Recurring  Investment 

•  Must  Break  the  Block  Upgrade  Paradigm  Made 
Necessary  by  the  Tight  Coupling  Between  OFPs  and 
Specific  H/W  Configurations 

•  Assembly  Language  OFPs  Have  Become  Increasingly 
Unstructured  Through  Many  Upgrade  Iterations 
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OSCAR  IPT  Open  System  Lesson  Learned  Analysis 


•  Represents  a  Snapshot-In-Time 

-  Where  We’ve  Been 

-  Where  We  Are 

-  Where  We’re  Going 

•  Compiled  by  the  Engineers  Working  the  Issues 

-  Analysis  of  Key  Impact  Areas 

•  Identifies  Current  Top  10  OSCAR  Lessons  Learned 

•  Provides  a  Basis  for  Future  Lessons  Learned 
Comparisons/Analysis 
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AV-8B  OSCAR  Principles 


•  Follow  US  DoD  Directive  For  Acquisition  Reform 

-  Apply  Revised  DoD  Directive  5000  (dated  15  Mar  96) 

-  Commercial  Business  Philosophy 

-  Performance  Based  Specs  vs  Procurement  Specs 

•  Insert  Commercial  Technologies 

-  COTS  Hardware 

-  COTS  Software  Development  Environment 

•  Reduce  Life  Cycle  Cost 

•  Apply  Open  System  Architecture 

-  Emphasis  on  Non-Proprietary  Hardware  and  Software 

-  Object  Oriented  Design  and  High  Order  Language 

-  Software  Independent  of  Hardware 

•  Increase  Allied  Software  Development  Workshare 
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Review  of  Early  Expectations 


•  OSCAR’s  Goals 

-  Reduce  Life  Cycle  Support  Cost  of  Software  Upgrades 
(Cost  Savings  to  be  Realized  during  3rd  Block  Upgrade) 

•  Shortened  OFF  Development  Cycle 

•  Reduce  Rework  in  Dev  Cycle  &  DT/OT 

•  Reduce  Regression  Testing  in  OC1.2 
(OC1. 1  set  baseline) 

-  Leverage  Commercial  Technology 

-  Incorporate  an  Open  Architecture  Concept 

-  No  Reduction  in  System  Performance 
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Review  of  OSCAR  Open  System 

Assumptions 


•  Implementation  of  Open  Systems  H/W  and  S/W  Requires 
Up-Front  Investment 

-  Recoupment  Within  2-3  Updates  to  the  S/W 

•  Open  System  Computing  H/W  is  Based  on  Commercial 
Standards 

-  Promotes  Competition 

-  Takes  Advantage  of  Commercially  Driven  Requirements  for  Technology 
Insertion 

•  LCC  Analysis  Shows  a  30-40%  Cost  Reduction  in  Core 
Computing  H/W  and  S/W  Development  but  not 
necessarily  applicable  to  System  Integration/Ti est  of 
Muiti-Sys  Block  Upgrades 
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Review  of  OSCAR  Open  System 
Assumptions  (cont.) 


•  OSCAR  and  Open  Systems  Computing  Does  Not  Affect 
Tasks  Associated  with  the  Airframe  or  Flight 
Qualification  of  New  Weapons/Capabilities 

•  Two-Level  Maintenance  Concept  Philosophy  Will 
Reduce  LCC  and  Increase  Operational  Availability 

•  OSA  provides  Arch  for  a  Plug-and-Play  Trainer  Concept 

•  With  OSCAR  as  First  Large  Scale  Implementation  of 
Open  Systems  and  Object  Oriented  S/W: 

-  Reluctance  to  Fully  Realize  the  Cost  Benefits  Until  OSCAR  is  Fielded 
and  all  the  Data  Collected  and  Analyzed 
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Review  of  OSCAR’S  Open  System 
Assumptions  (cont.) 


•  OSCAR’s  Open  System  Architecture  Will  Make 
Incremental  Upgrades  Possible  by  Decoupling  H/W  and 
S/W  (i.e.,  MSC-750-G4) 

•  Commercial  Off-The-Shelf  Products  can  be  Directly 
Incorporated  with  Minimal  Development  Costs 

-  Multi-Vendor  Support  Ensures  Competitive  Procurement  Costs 

•  Software  LCC  Savings  are  Derived  from  the  High 
Degree  of  Modularity  Envisioned 

-  Less  Than  Half  the  Regression  Test  and  Re-Qual  Effort  of  Today 
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Data  &  Metrics  Currently  Collected 


•  SPI 

•  CPI 

•  Requirements  -  System  &  software  levels,  stability  index 

•  SLOC  -  Estimates  vs.  actuals,  productivity  factor 

•  Classes 

•  Peer  Review 

•  TWO  -  Development  &  ground  test  execution 

•  Flight  Test  -  flights,  test  points,  analysis 

•  Problem  Reports  -  various  flavors 

•  Throughput  &  Memory  Spare 

•  Hardware  Performance 

•  Risk 
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Initial  Expectations  for  Metrics 


SPI  -  Identify  an  immediate  schedule  problem 


CPI  --  Control  overspending,  identify  underruns 

System  &  Software  Requirements  --  Track  the  development 
to  plan  and  identify  any  Growth 

Requirements  Stability  --  Control  requirements  growth 

SLOC  Actuals  vs.  Estimated  --  Control  growth  and  ‘gold- 
plating’ 


Software  productivity  (Manhrs/SLOC)  --  Improve  efficiency 
within  which  software  is  produced 


Classes  Actuals  vs.  Planned  To  Date  -  Indication  of 
performance  to  schedule 
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Initial  Expectations  of  Metrics 


•  TWD  Development  &  Ground  Test  -  Readiness  of 
test  team  to  support  system  level  test  phase 

•  Problem  Reports  --  Quality  of  the  software  & 
where  are  problems  found 

•  Throughput/Memory  —  Keep  software  within  the 
bounds  of  hardware  performance 

•  Risk  --  Control  risks  &  be  prepared  to  act  quickly 
if  they  materialize 
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What  Metrics  Actually  Provided 


•  SPI  --  Watch  The  Details 

-  Lower  level  problems  are  masked  within  larger  cost 
accounts 

-  Top-level  SPI  can  mask  lower  level  account  SPI 
difficulties 
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What  Metrics  Actually  Provided 


•  CPI  —  New  functionality  Costs  More  Than 
Legacy 


0SCAR-TDL1 2-003-V1 


What  Metrics  Actually  Provided 


•  System  Requirements  -  No  Changes  Resulting  From 
OO/C++  Development 

-  Level  Of  Detail  &  Complexity  Commensurate  With  Assembly 

-  OO  Makes  Traceablity  To  Code  Is  Difficult  (see  other  chart) 

•  Requirements  Stability  --  good  to  show  what’s  moving 
through  the  system,  but  don’t  really  know  how  many 
requirements  and  corresponding  code/tests  are  affected 
(traceability) 

•  Risks  -  hard  to  maintain  a  monthly  review  juggling 
schedules,  but  good  tool  to  keep  on  top  of  issues,  when 
High  risks  are  identified  -  resources  are  focused  on  them 

-  Engineers  tend  to  set  risks  at  HW/SW  detail  level  and  not  see  the 
top  level  System  Functionality  High  Risks 
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What  Metrics  Actually  Provided 


•  Throughput  Usage 

-  OO ,  COTS  OS  makes  throughput  consumption  difficult 
to  predict 


MSC/AMC  Throughput  Utilization 
Actual  Throughput  Consumed,  Estimate  by  Iteration,  and  Estimate  at  Complete 
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What  Metrics  Actually  Provided 


MSL/Tvlvfc  NVkAM  Memory  Utffizalion 

Actual  Memory  Consumed,  Estimate  by  Iteration,  and  Estimate  at  Complete 


MSC/AMC  RAM  Memory  Utilization 

Actual  Memory  Consumed,  Estimate  by  Iteration,  and  Estimate  at  Complete 


•  Memory  Usage 


Actual  Memory  Consumed,  Estimate  by  Iteration,  and  Estimate  at  Complete 


-  Consumption  can  be  predictably  scaled  from  assembly 
226/00  language  implementation  osu.Logan.ppt 
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What  Metrics  Actually  Provided 


Problem  Reports  -  Open/Closed/Rejected 

-  OO/C++  enables  trained  developers  with  Tools  to  rapidly 
diagnose  and  correct  anomalies. 
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What  Metrics  Actually  Provided 


•  Problem  Reports  -  Where  Found 

-  DTE  Saves  Time  &  Money 

-  Provides  a  “ Software  Test  Facility”  on  every 
desktop 

-  Less  problems  found  in  flight  than  Legacy  OFP 


2/26/00 


OSLLLogan.ppt 


OSCAR-TDL1 2-003-V1 


2/26/00 


What  Metrics  Actually  Provided 


SLOC 


-  Not  very  useful 

•  Some  code  “auto” -generated  by  4th  generation  tools 

-  Poor  unit  for  estimating  resources  required 

MSC  OC1.1  SLOC  Count 


onnnnn 


- Source  Lines  of  Code  (Actual)  -  -  -  -  Iteration  5  Plan  ...  -  Iteration  6  Plan 
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What  Metrics  Actually  Provided 
•  Classes 

-  Best  measure  of  development  progress 

•  Similar  to  function  points 

•  SLOC  difficult  to  estimate 

MSC  Development  Status  History 


Date 


- Actual  Total  - Planned  Total  - End  Total 
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What  Metrics  Actually  Provided 


Risk 

-  Good  tool  to  keep  on  top  of  issues  but  can  bring  too 
much  Political  help 

•  When  high  risks  are  identified  --  resources  are 
focused  on  them 

-  Discipline  of  regular  periodic  review  is  important  to  keep 


the  focus 


Transition  to  AMC&D 


A/G  solutions 


•  Compare  flight  test  results  to  manned 


Consider  New  Risks 
•16  OFP  Release 


•  Measure  AV-8  AMC  unit  #1  thruput 
performance 

•  Begin  integration  with  initial  delivery 
of  s/w  &  h/w  (with  750  processor) 

•  Verify  sufficient  16  thruput  for 
alternate  configuration  (RAM  with 


Supplier  support  during  flight  test 

•  Get  Raytheon  on  contract 


1  2  3  4  5 

Consequence  (C0) 


^lntearat^MRAAM^3^nt^AV-8B 

•  Confirm  MSC/Radar  timing 
analysis  on  AMC 


Tactical  mode  sensors 


Provide  programmable  filter 
constants  for  flight  test 
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Summary  of  OS  Lessons  Learned 
For  Currently  Collected  Metrics 

•  SPI  --  Watch  The  Details 

•  CPI  —  New  functionality  Costs  More  Than  Legacy 

•  System  Requirements  -  No  Changes  For  Assembly 
-  Traceablity  To  Code  Is  Difficult 

•  TWD  Development  --  Same  as  in  Traditional 
Development 

•  SLOC  count  --  Not  as  Useful  for  OO/C++ 
Development  Tracking 

•  Classes  --  Good  Indicator  of  Development  Progress 
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Summary  of  OS  Lessons  Learned  For 
Currently  Collected  Metrics 


•  Problem  Reports  -  Total  --  OO/C++  a  Benefit  to 
Problem  Resolution 

•  Problem  Reports  -  Where  found  --  DTE  Saves 
Time  &  Money 

•  Throughput  Usage  -  OO,  COTS  Makes  Prediction 
Difficult 

•  Memory  Usage  -  Scaleable  from  Legacy 
Development 

•  Risk  -  Good  Tool  to  Focus  Attention  &  Resources , 
if  Risk  Identification  doesn’t  get  too  Political 
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Technology  Challenges 


COTS  supports  the  code/debug/unit  test  stages  of  development  well 
but  many  Voids  still  exist: 

•  “Front  end ”  of  process 

-  Model-based  tools  for  requirements/design  capture 

-  Automated  configuration  and  integration  of  components 

•  “Back  end”  of  process 

-  Simulation-based  testing 

•  Support  for  hard  real-time  embedded  systems  is  limited 

-  Quality-of-service  requirements  expression/guarantees 

•  Legacy  system  constraints 

-  Infusing  new  technology  into  resource-limited,  “closed” 
systems 

•  High  Integrity  System  development  technologies 
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Cultural  Challenges 


•  Acquisition  culture  presents  impediments  as  well 


-  “Silo”  approach  to  planning/funding  system 
modernization 

-  “Wasn  ’ t  in  vented  here  ”  mindset  in  programs 

-  Inability  to  trade  front-end  investment  for  life-cycle 
returns,  even  when  business  case  is  compelling 

-  Synergy  with  COTS  industry  will  always  be  limited 
without  cultural  transformation 

-  Support  structure  based  on  single  fielded  configuration 

-  T&E  community  resistance  to  tailored  re-qualification 


No  incentive  for  multi-platform  development 


2/26/00 


28 


OSLLLogan.ppt 


OSCAR-TDL1 2-003-V1 


OSA  Lessons  Learned  -  Standards 


Goal:  Use  Widely  Accepted  Commercial  Standards 

•  Standardize  Module  Form,  Fit,  Function  and  Interface 
(F3I)  to  Allow  Functional  Performance  Upgrades 

•  USE  COTS  Standards  for  Networks,  Processors, 
Memory,  and  Operating  System 

Reality:  Existing  Commercial  Standards  Do  Not  Typically 
Accommodate  Aerospace  Requirements 

•  Real  Time  Operation  -  Flight  Dynamics 

•  Memory  Partitioning  for  Fault  Containment 

•  Built-In-Test 

Solution:  Modify  Commercial  Standards  Through  Active 

Participation  in  Standards  Bodies 

•  ANSI  Fibre  Channel  Avionics  Environment  (FC-AE) 

•  Modify  Commercial  STD  Common  Object  Request 
Broker  Architecture  (CORBA)  for  Real-Time  Operation 

•  Add  Service  Layers  on  Top  of  Commercial  Software 
Infrastructure 
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OSA  Lessons  Learned  -  Specifications 


Goal:  Focus  on  Specifying  Functional/Performance 

Requirements  versus  “How  To” 

•  Use  Commercial  Specs  Wherever  Possible 

•  Use  Tailored  Mil-Specs 

•  Eliminate  Unnecessary  “How  To”  specs 

Reality:  It  is  Difficult  to  Prevent  Engineers  (Boeing,  Customer, 
and  Supplier)  From  Diving  Down  Into  Too  Much  Detail 

•  Commercial  Specifications  may  not  match  Aerospace 
requirements 

•  Additional  effort  needed  to  ensure  Performance 
Levels  and  interoperability  Are  Achievable 

Solution:  Need  to  get  a  Better  Handle  on  the  High  Level 

Performance  Requirements 

•  Develop  benchmark  application  program  to  validate 
memory  and  throughput  for  COTS  processors 

•  Using  a  “Performance  Prediction  Team”  to  Conduct 
Simulation  and  Modeling  of  Key  System  Attributes. 

•  Evaluate  Lab  Prototype  H/W  to  Gather  Data. 
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COTS  Lessons  Learned 


•  COTS  May  Not  Work  As  Well  For  Your  Application  As  The 
Application  For  Which  It  Was  Developed 

•  COTS  Frequently  Has  Surprises,  Especially  With  Little  Used 
Features 

•  COTS  Documentation  May  Be  Lacking,  Or  Will  Not  Tell  You 
How  It  Will  Work  In  Your  System 
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Lessons  Learned  -  Diagnostics 


•  Diagnostics  Processes/Tools  must  better  address 
False  Alarm  Rate 

•  Supplier  must  better  understand  Total 
Diagnostics  Requirements 

-  Fault  Coverage 

-  Fault  Isolation 

-  False  Alarms 

-  Failure  Reporting  &  Recording 

•  Diagnostic  System  must  have  integrated  on¬ 
board  and  off-board  capability  that  can  be 
updated  in  a  timely  manner 
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Lessons  Learned  -  Prototyping 


•  Early  And  Frequent  Prototyping  Required 
Throughout  The  Program 

•  Develop  Software  Incrementally  Utilizing  Daily 
Builds 

•  Complex  Functionality  needs  to  be  partitioned  and 
implemented  early 

•  Verify  Design  And  Ensure  A  Pi's  Meet  Needs  Of 
User 

•  Verify  Software  And  Hardware  Performing  As 
Expected 


No  New  Lessons  from  Legacy  Developments 
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Object  Oriented  Design  in  a  Functional 
Decomposition  World 
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Early  Returns  -  Measured  Benefit 


Cumulative  Software  Development  Productivity 


Key  Sources  of  Gain: 

•  Reuse  (of  all  types) 

•  COTS  Tools 

•  Change  Containment 

•  Desktop  Testing 

•  High  Order  Language 
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Historical  F/A-18  Legacy  Cost 


Historical  F-15  Legacy  Cost: 


Goal 


X*- 

F-15 


F/A-18 


Measured  Software  Development  Affordability  Improvement 
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S/W  Development  Productivity 
(Hand  plus  Rose  Generated  Code) 
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Lesson  Learned  -  OSCAR 

Hardware 
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Qual  Test 


•  The  following  environmental  qual  tests 
have  been  completed : 

MSC  &  WMC 

-  Temp- Alt 

-  Vibration 

-  EMIC 

-  Acoustic  Noise 

-  Loads 

-  Shock 

-  Humidity 

-  Salt 

-  Exp  Atmosphere 

-  Sand  &  Dust 
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Qual  Test  Cont’d 


•  COTS  hardware  did  Well. 

-  No  problems  with  off-the-shelf  DY-4  Processor 
board  (one  capacitor  failure  in  RDT. 


•  No  problems  with  plastic  parts 
(PEMS) 

-  Hardware  with  plastic  parts  were  exposed  to 
MiL-STD-810  Humidity  and  Salt-Fog 
environments  in  two  WRA’s  with  no  failures. 

-  Was  a  major  concern  of  some  people  early  in 
the  program. 
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Reliability 


•  Reliability  experience  to  date  with  COTS  hardware  has  been 
good. 

•  Reliability  Development  Testing  (RDT)  done  on  three  WRAs. 

-  WMC  - 1,000+  hours 

-  MSC#1- 1,000+ hours 

-  MSC  #2  - 1,000+  hours 

•  One  capacitor  failure  on  COTS  board,  Root  cause  unknown. 

•  One  commercial  grade  capacitor  failed  on  another  SRA. 
Switching  to  a  MIL-SPEC  capacitor. 

•  Other  failures  occurred,  but  unrelated  to  COTS  hardware. 
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Memory  and  Throughput 


•  OOD  is  a  big  resource  consumer : 

•  The  F-15  Central  Computer  OFP  had  already  been 
converted  from  an  assembly  language  to  a  HOL 
(Ada)  in  the  early  1990’s . 

•  Felt  comfortable  with  initial  OSCAR  estimates 
based  on  complexity  of  the  F-15  aircraft  versus 
the  A  V-8B,  a  six  processor  solution  (on  the  F-15) 
versus  a  single  processor,  and  the  continued 
growth  in  available  throughput  in  commercial 
processors . 


However,  a  4x  estimate  turned  into  a  40x  reality 
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F-15  Mission  Computer  Memory  Utilization 
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F-15  and  AV-8B  Mission  Computer  (pre-OSCAR) 

Memory  Utilization 


^  ^  ^  ^  ^  ^  ^  ^  ^  ^ 


■  F-15 

■  AV-8B 
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F-15  and  AV-8B  Mission  Computer  memory 

Utilization 


^  ^  ^  ^  ^  ^  ^ 


■  F-15 

■  AV-8B 


Memory  and  Throughput  Conclusions 


•  Use  of  OOD  has  a  tremendous  impact  on 
Memory  usage. 

•  Believe  throughput  impact  is  even  greater, 
although  more  difficult  to  compare. 

•  Lesson  Learned  -  Use  of  OOD  adds  an 
order  of  magnitude  (or  more)  to  memory 
and  throughput  requirements. 
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Tools  Lessons 
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OSA  Lessons  Learned  -  Tools 


•  Not  All  Commercial  Tools  Scale  To  Large 
Development  Programs 

•  Interoperability  Of  Commercial  Tools  Must  Be 
Evaluated  Prior  To  Selection 

•  Keep  Up  With  New  Tool  Versions  To  Maintain 
Vendor  Support 

•  Plan  Tool  Transitions 

•  Utilize  Dedicated  Tool  Engineers 
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Tool  Compatibility 


Rapid  Version  Rolls 

98,  98i,  2000) 


Tool  du  J  our 
Attitude 

(PVCS  vs  ClearCase) 


S/SEE  Management 
Across  Geographically 
and  Organizationally 
Separated  Development 
Sites  requires  tight  Tool  CM 


OSLLLogan.ppt 


OSCAR-TDL1 2-003-V1 


Desktop  Test  Environment 


Common  Test  Language  System  BE]  B 


FA18_CautionsAndWarnings  -  Microsoft  Developer  Studio 


File  Edit  Search  SUT-Debug  Show  Watch  Options  Window  Help 

|  Break  |  Exit  |  EWatch  Update  | 


Test  File  :  C:\DTE\RELEflSE\BIN\F18flDDH0DELS0FPSHE 

Start  Time  :  26  August  1997  09:50:08 

End  Time  :  26  August  1997  09:53:24 

Elapsed  Time  :  00:03:16 

Fatal  Errors 
Errors 
Warnings 


-  jEMe  Edit  View  insert  Eroject  Build  lools  Window  Help 
#  ®  *  !  >1  ■'ft  %  1&  Hi  [Hi  |  *  t1}  O1  ■*{>  |  jp 

IQ  ^  [gelchain 


21 


TF  RUN"C :\DTE\Release\bin\F18AddModelsOFPShell .ctl" 


sin 


-  source 

i|A8CSC.cpp 
11|  A8CSC_CNICautions( 

|  aiA8CSC_DeviceHandl 
I  A  A8CSC_FailuresConv| 
H  ASCSCVoiceAURCe 
1|1  A8CSC_VoiceCuesCc 
;  ||  A8LogicalVoiceDevic" 
I  ||  A8VoiceAlert.cpp 
||  A8VoiceType.cpp 
|  ||  ABool64.cpp 
|  n  ABool64Converter.cp 
n  ABoolConverter.cpp 


n 


C:\DTE\Release\bin\F1 8AddModelsOFPS 


#include 

#include 


‘UIO_Data: 

'UI01553D. 


#include  "A8CSC .h' 


void  main() 

{ 


Loaded  'C:\WINNT\system32\WS2_32.DLL',  no  match 
Loaded  symbols  for  'C:\WINNT\system32\MSVCRT.DLI 
Loaded  'C:\WINNT\system32\WS2HELP.DLL',  no  matclj 
Loaded  symbols  for  'C:\WINNT\system32\Msvcirtd 
Loaded  1 C : \WINNT\system32\RPCLTC 1 . DLL  1 ,  no  matcl| 
JjJ\  Build  \  Debug  /  Find  in  Files  1  \  Find  in  Files  2~7  [j 


Rapidly  design  once 
•Autogenerated  code 
•COTS  processors  &  tools 
•Developers  run  OFP 
at  their  desk 
•Reduces  time  and  cost 
•Enabled  by  hardware  and 
O/S  change  containment 


jB Start | jC:\  |  3  Clicker  |  aPFA18...|  DTE 


ajDTE  ...|  "^Cotn...  |  ;iBf18_...  [  jjjCAD...  [[Throttle  iiifitemp 


10:03  AM 
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Summary 
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Lessons  Learned  Summary 
(Most  Critical) 


•  COTS 

-  Use  Existing  Products 

•  Don’t  Push  Technology,  Follow  It  (Cost/Schedule\Risk) 

•  Use  Technology  Rolls  To  Satisfy  Growth,  Not  Baseline 
Requirements 

-  DOD  Programs  Have  Limited  Influence  On  Commercial 
Developments 

•  Very-Very-Small  Quantities  Compared  to  Industry 

-  COTS  Does  Well  In  Qualification  Testing 

•  Open  Systems  Design 

-  Cultivate/Develop  Multiple  Production  Sources  Up  Front 

-  Partition  Software  Workpackages  Along  Functional  Lines 
(Self  Contained  Packages) 
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Lessons  Learned  Summary  (Cont.) 
(Most  Critical) 


•  C++  /  00  Design 

-  Throughput  Is  Difficult  To  Estimate 

-  Scale  The  Software  To  the  EXISTING  Computer  Resources: 

•  Memory,  Throughput,  I/O 

-  In  Order  To  Reuse  Functional  Software  The  Top  Level 
Requirements  MUST  Be  The  Same 

-  Reused  Software  Will  Require  Significant  Rework 

-  Process  &  Procedures  Are  No  Substitute  For  A  Stable,  Well- 
Trained  Workforce 

-  Troubleshooting  Transient  Problems  Is  More  Difficult  in  COTS 
Environment 

-  Turnaround  On  Fixes  Is  Much  Quicker 

•  Functionality 

-  Document  And  Bound  All  Requirements 

-  Limit  New  Functionality  Until  After  Legacy  Is  Complete 

-  Be  Selective  in  Legacy  Problem  Fixing  During  Conversion 

•  Use  Multiple  Metrics  To  Identify  Problems 
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Priority  Order  of  the 
Top  10  OSCAR  Lessons  Learned 


1  --  Document  And  Bound  All  Requirements 

2  --  Reused  Software  Will  Require  Significant  Rework 

3  --  Process  &  Procedures  Are  No  Substitute  For  A  Stable  Well  Trained 

Workforce 

4  --  Throughput  Is  Difficult  To  Estimate  (00) 

5  --  Use  Existing  Products  (COTS) 

6  --  Use  Multiple  Metrics  To  Identify  Problems 

7  --  DOD  Programs  Have  Limited  Influence  On  Commercial  Developments 

8  --  Troubleshooting  Transient  Problems  Is  More  Difficult 

9  --  In  Order  To  Reuse  Functional  Software  The  Top  Level  Requirements 

MUST  Be  The  Same 

10--  Partition  Software  Workpackages  Along  Functional  Lines  -  (Self  Contained 
Packages) 
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Summary 


•  How  Are  We  Doing  with  Respect  to  Earlier  Expectations? 

-  LCC  savings  and  schedule  improvements  will  not  be  realized 
until  2nd  and  3rd  upgrades 

-  Thruput  estimates  were  off  by  an  order  of  magnitude 

•  Where  Are  We  Going  with  the  Open  Systems  Approach? 

-  Boeing  Company  roadmap  for  all  legacy  and  future  A/C 
system  upgrades 

•  Where  Are  We  Going  with  Metrics  Collection? 

-  Classes  planned-vs-actuals  is  the  best  metric  for  program 
progress  indicator 

-  Will  continue  to  collect  thru  0C1.3  to  set  baseline 

•  What  Are  We  Going  to  “Do”  with  Lessons  Learned  Metrics? 

-  Compare  to  legacy  systems  metrics(  where  available)  and 
produce  /  quantify  data  to  establish  baseline  for  F/A-18  &  JSF 
systems  development 

-  Incorporate  lessons  learned  into  Boeing-wide  training 
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The  Next  Step 


Answer  5  Questions 
(Based  On  OSCAR  Experiences) 

1  -  How  Fast  Can  The  Investment  Costs  Be  Recaptured? 

2  -  Is  00/C++  Software  Transparent  To  Hardware? 

3  —  What  is  the  Ratio  Of  New  Functionality  Development 

Costs  Of  00/C++  vs.  Assembly 

4  --  Does  00/C++  Software  Reduce  Retest? 

5  —  Is  COTS  Less  Expensive? 
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The  Next  Steps  -  Develop  A  Plan 


ueveiop  a  rian/Process  to  uollecT/ueneraie  DaiaA  that  win 

Support  the  Determination  of: 

1  -  Actual  Cost  Of  OSCAR  Software  Conversion 

•  Use  As  Basis  For  Determining  Investment  Cost 

•  Factor  Out  New  Functionality 

•  Requirements  through  Fleet  Release 

•  Compare  Against  Original  Estimates 

-  If  Different,  Why? 

2  -  Actual  Cost  Of  New  Hardware  (WMC  /  AMC) 

•  Development  Of  Boxes 

-  Use  As  Basis  For  Determining  Investment  Cost 

•  Unit  Production  Costs 

•  Compare  Against  Predictions 

•  Compare  Against  Dedicated  Mil  Spec.  Box  (Non-COTS) 

3  -  Was  COTS  Less  Expensive? 

•  Why  or  Why  Not? 
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The  Next  Steps  -  Develop  A  Plan 


Develop  A  Plan/Process  to  Collect/Generate  Data*  that  will 

Support  the  Determination  of: 

4  -  Actual  Costs  Of  new  Functionality 

•  AMRAAM/13C  (0C1.1) 

•  JDAM,  HQ/SG  (0C1.2) 

5  -  Comparsion  With  Assembly  Language  Version 

•  Was  It  Cheaper  to  Develop?  To  Test? 

-  Why? 

6  -  “Will  OO  &  C++  Cause  Less  Retest  In  Subsequent  OFPs?” 

•  How? 

-  Generate  An  OC1.2  Metric  To  Measure  Unplanned  Fixes  To  Legacy 
Caused  By  New  Functionality 

7  -  Costs  Associated  With  Migrating  OSCAR  OFP  To  New  Processors 

•  603e  to  750 

•  750  to  G4 

•  Was  Hardware  Transparent  to  Applications  OFP? 

-  If  Not  then  Why? 

-  Identify  Issues 


2/26,00  57  OSLLLogan.ppt 

oscar  TDLtlAfote;  Some  Data  May  Not  Be  Available  Until  After  The  Completion  Of  OC1. 1  &  AMC&D 


The  Next  Steps  -  Determine  the  Pay  Back 


•  Using 

-  The  Initial  Investment  Costs 

-  Follow  On  New  Development  Costs 


•  Determine 

-  How  Much  Software  Must  Be  Written  To  Pay  Back  Initial 
Investment 
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Bold  Stroke 

Open  Systems  Lessons  Learned 
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